CLOI-NET: Class segmentation of industrial facilities' point cloud datasets

Shape segmentation from point cloud data is a core step of the digital twinning process for industrial facilities. However, it is also a very labor intensive step, which counteracts the perceived value of the resulting model. The state-of-the-art method for automating cylinder detection can detect cylinders with 62% precision and 70% recall, while other shapes must then be segmented manually and shape segmentation is not achieved. This performance is promising, but it is far from drastically eliminating the manual labor cost. We argue that the use of class segmentation deep learning algorithms has the theoretical potential to perform better in terms of per point accuracy and less manual segmentation time needed. However, such algorithms could not be used so far due to the lack of a pre-trained dataset of laser scanned industrial shapes as well as the lack of appropriate geometric features in order to learn these shapes. In this paper, we tackle both problems in three steps. First, we parse the industrial point cloud through a novel class segmentation solution (CLOI-NET) that consists of an optimized PointNET++ based deep learning network and post-processing algorithms that enforce stronger contextual relationships per point. We then allow the user to choose the optimal manual annotation of a test facility by means of active learning to further improve the results. We achieve the first step by clustering points in meaningful spatial 3D windows based on their location. Then, we apply a class segmentation deep network, and output a probability distribution of all label categories per point and improve the predicted labels by enforcing post-processing rules. We finally optimize the results by finding the optimal amount of data to be used for training experiments. We validate our method on the largest richly annotated dataset of the most important to model industrial shapes (CLOI) and yield 82% average accuracy per point, 95.6% average AUC among all classes and estimated 70% labor hour savings in class segmentation. This proves that it is the first to automatically segment industrial point cloud shapes with no prior knowledge at commercially viable performance and is the foundation for efficient industrial shape modeling in cluttered point clouds

Detecting healthy concrete surfaces

Teams of engineers visually inspect more than half a million bridges per year in the US and EU. There is clear evidence to suggest that they are not able to meet all bridge inspection guideline requirements due to a combination of the level of detail expected, the limited time available and the large area of bridge surfaces to be inspected. Methods have been proposed to address this problem through damage detection in visual data, yet the inspection load remains high. This paper proposes a method to tackle this problem by detecting (and disregarding) healthy concrete areas that comprise over 80-90% of the total area. The originality of this work lies in the method’s slicing and merging to enable the sequential processing of high resolution bridge surface textures with a state of the art classifier to distinguish between healthy and potentially unhealthy surface texture. Morphological operators are then used to generate an outline mask to highlight the classification results in the surface texture. The training and validation set consists of 1,028 images taken from multiple Department of Transportation bridge inspection databases and data collection from ten highway bridges around Cambridge. The presented method achieves a search space reduction for an inspector of 90.1% with a risk of missing a defect patch of 8.2%. This work is of great significance for bridge inspectors as they are now able to spend more time on assessing potentially unhealthy surface regions instead of searching for these needles in a mainly healthy concrete surface haystack.This work is partly funded by Trimble Inc. and by the European Union’s Seventh Framework Programme for research, technological development and demonstration under grant agreement no. 31109806.0007. SeeBridge is co-funded by Funding Partners of the ERA-NET Plus Infravation and the European Commission. The Funding Partners of the Infravation 2014 Call are: Ministerie van Infrastructuur en Milieu, Rijkswaterstaat, Bundesministerium für Verkehr, Bau und Stadtentwicklung, Danish Road Directorate, Statens Vegvesen Vegdirektoratet, Trafikverket – Trv, Vegagerðin, Ministere de L’ecologie, du Developpement Durable et de L’energie, Centro para el Desarrollo Tecnologico Industrial, Anas S.P.A., Netivei Israel – National Transport Infrastructure Company Ltd. and Federal Highway Administration USDOT

Digital twinning of existing reinforced concrete bridges from labelled point clusters

The automation of digital twinning for existing reinforced concrete bridges from point clouds remains an unresolved problem. Whilst current methods can automatically detect bridge objects in point clouds in the form of labelled point clusters, the fitting of accurate 3D shapes to point clusters remains largely human dependent largely. 95% of the total manual modelling time is spent on customizing shapes and fitting them correctly. The challenges exhibited in the fitting step are due to the irregular geometries of existing bridges. Existing methods can fit geometric primitives such as cuboids and cylinders to point clusters, assuming bridges are comprised of generic shapes. However, the produced geometric digital twins are too ideal to depict the real geometry of bridges. In addition, none of the existing methods have explicitly demonstrated how to evaluate the resulting Industry Foundation Classes bridge data models in terms of spatial accuracy using quantitative measurements. In this article, we tackle these challenges by delivering a slicing-based object fitting method that can generate the geometric digital twin of an existing reinforced concrete bridge from four types of labelled point cluster. The quality of the generated models is gauged using cloud-to-cloud distance-based metrics. Experiments on ten bridge point cloud datasets indicate that the method achieves an average modelling distance of 7.05 cm (while the manual method achieves 7.69 cm), and an average modelling time of 37.8 seconds. This is a huge leap over the current practice of digital twinning performed manually

Understanding the problem of bridge and tunnel strikes caused by over-height vehicles

A bridge or tunnel strike is an incident in which a vehicle that is taller than the clearance underneath the structure (over-height), typically a lorry or double-decker bus, collides with the structure causing damage. This can lead to injuries, fatalities and/or, in worst case scenario, train derailments. Bridge and tunnel strikes are costly and expensive. The annual maintenance costs to repair and service the structure have been reported to range in the tens-to-hundreds of thousands (£) while the average cost per strike ranges between £5,000 to £25,000. In this paper, we present a comprehensive synthesis of the nature and scope of the problem of bridge and tunnel strikes, followed by the current state of practice and current state of research. Bridge and tunnel strikes still occur with high frequency, and prevention systems (passive, sacrificial and active) available on the market are often too expensive. Bridge-owners are seeking an affordable yet reliable system that is cheap enough for widespread installation without compromising the accuracy and performance of such a system.This material is based upon work supported by Transport for London.This is the final version of the article. It first appeared from Elsevier via http://dx.doi.org/10.1016/j.trpro.2016.05.48

Trajectory-based worker task productivity monitoring

Over the past decades labour productivity in construction has been declining. The prevalent approach to estimating labour productivity is through an analysis of the trajectories of the construction entities. This analysis typically exploits four types of trajectory data: a) walking path trajectories, b) dense trajectories (posture), c) physiological rates such as heart rate (beats/minute) and respiratory rate (breaths/minute), and d) sound signals. The output of this analysis is the number of work cycles performed by construction workers. The total duration of these cycles is equal to the labour input of a task. However, all such methods do not meet the requirements for proactive monitoring of labour productivity in an accurate, non-obtrusive, time and cost efficient way for multiple workers. This paper proposes a method to address this shortcoming. It features a promising accuracy in terms of calculating the labour input.ICASE studentship award, supported by EPSRC and LAING O'ROURKE PLC under Grant No. 13440016

Prioritising Object Types of Industrial Facilities to Reduce As-Is Modelling Time

The cost of modelling existing industrial facilities is currently considered to counteract the benefits of the model in managing and retrofitting the facility. 90% of the modelling cost is typically spent on labour for converting point cloud data to the final model, hence reducing the cost is only possible by automating this step. Previous research has successfully validated methods for modelling specific object types such as pipes. Yet modelling is still prohibitively expensive. We tackle a part of this issue by identifying the most frequent object types that require modelling in industrial plants to guide future work aimed at automating the tedious current practice. We determine a priority list of the object types in these facilities based on their frequency of appearance (%) and intent of modelling. A parametric study based on Outer Diameter (OD) then finds the most frequent OD ranges for these objects. The results indicated that steel sections were the most frequent object type encountered in all case studies

Road Design Layer Detection in Point Cloud Data for Construction Progress Monitoring

Poor performance in transportation construction is well-documented, with an estimated $114.3 billion in global annual cost overrun. Studies aimed at identifying the causes highlighted traditional project management functions like progress monitoring as the most important contributing factors. Current methods for monitoring progress on road construction sites are not accurate, consistent, reliable, or timely enough to enable effective project control decisions. Automating this process can address these inefficiencies. The detection of layered design surfaces in digital as-built data is an essential step in this automation. A number of recent studies, mostly focused on structural building elements, aimed to accomplish similar detection but the methods proposed are either ill-suited for transportation projects or require labelled as-built data that can be costly and time consuming to produce. This paper proposes and experimentally validates a model-guided hierarchical space partitioning data structure for accomplishing this detection in discrete regions of 3D as-built data. The proposed solution achieved an F1 Score of 95.2% on real-world data confirming the suitability of this approach.This research is made possible through funding from the United States Air Force and the Cambridge Commonwealth and International Trust. The authors express gratitude to the Trimble Corporation for their support in lending equipment and expertise to the data collection operation

Multistep Explicit Stereo Camera Calibration Approach to Improve Euclidean Accuracy of Large-Scale 3D Reconstruction

The spatial accuracy of point clouds generated by stereo image-based 3D reconstruction algorithms is very sensitive to the intrinsic and extrinsic camera parameters determined during camera calibration. The existing camera calibration algorithms induce a significant amount of error due to poor estimation accuracies in camera parameters when they are used for large-scale scenes such as mapping civil infrastructure. This leads to higher uncertainties in the location of 3D points, and may result in the failure of the whole reconstruction process. This paper proposes a novel procedure to address this problem. It hypothesizes that a set of multiple calibrations created by videotaping a moving calibration pattern along a specific path can increase overall calibration accuracy. This is achieved by using conventional camera calibration algorithms to perform separate estimations for some predefined distance values. The result, which includes multiple sets of camera parameters, is then used in the Structure from Motion process to improve the Euclidean accuracy of the reconstruction. The proposed method has been tested on infrastructure scenes and experimental analyses indicate more than 25% improvement in the spatial accuracy of 3D points.This is the accepted manuscript. The final version is available from ASCE at http://dx.doi.org/10.1061/(ASCE)CP.1943-5487.000045

A benchmark framework of geometric digital twinning for slab and beam-slab bridges

We devise, implement, and benchmark a framework LUKIS to automate the process of geometric digital twinning for existing slab and beam-and-slab bridges. LUKIS follows a top-down strategy to detect and twin bridge concrete elements in point clouds into an established data format Industry Foundation Classes. Existing software packages require modellers to spend many labour hours in generating shapes to fit point cloud sub-parts. Previous methods can generate surface primitives combined with rule-based classification to produce cuboid and cylinder models. While these methods work well in synthetic datasets or simplified cases, they encounter challenges when dealing with real-world point clouds. We tackle this challenge by investigating the entire workflow of geometric digital twinning for bridges and proposing LUKIS to auto-generate bridge objects without needing to generate low-level surface primitives. We implement LUKIS on a single software platform. Experiments demonstrate its ability to rapidly twin geometric bridge concrete elements. Compared to manual operation, LUKIS reduces the overall twinning time by at least 95.4% while the twinning quality (spatial accuracy) is improved. It is the first framework of its kind to achieve the geometric digital twinning for primary concrete elements of bridges on one platform. It lays foundations for researchers to generate semantically enriched digital twins